Resource management and performance control for staged design-based services

The staged architecture has emerged as an approach to implement highly concurrent Internet services. Staging means that the functionality of the server code is broken down into computational stages with each stage performs some aspect of request processing. A client request would then be processed along a pipeline of these stages. This architecture allows services to behave well and gracefully handle overload, in addition to increase code modularity and simplify service design. However, Staged architecture has introduced other design challenges related to resource management and performance control. A bottleneck stage in the work-flow of requests processing will limit the overall system throughput even though other stages are isolated from this bottleneck stage and can support a higher performance. For this reason care must be taken to avoid bottlenecks. A solution to adjust the throughput is to allocate more resources to a stage if it is becoming a bottleneck, this in turn may force other stages into becoming a bottleneck. Since all stages are competing for the same resources additional effects may take place and give rise to instability or oscillations. Staged design was originally introduced as a programming abstraction to improve memory accesses behavior of highly concurrent Internet servers by implementing cohort scheduling policies which batch the execution of requests at each stage. As a result existing scheduling policies in staged design-based applications are mechanisms to increase the benefit from cache locality within the individual stages, rather than to balance resource allocation to avoid bottleneck stages and control the system performance. Consequently, dynamic changes in stages requirements lead to instability and oscillations in performance under different load conditions. In addition, achieving a target performance in such systems is a hard job and often depends on manual parameters tuning by expert administrators or benchmarks experiments. To address these challenges, this thesis suggests a three-layers control architecture for resource management and performance control of staged applications based on the Staged Event Driven Architecture (SEDA), which is the